A general framework for the evaluation of symbol recognition methods
Identifieur interne : 005543 ( Main/Exploration ); précédent : 005542; suivant : 005544A general framework for the evaluation of symbol recognition methods
Auteurs : E. Valveny [Espagne] ; P. Dosch [France] ; Adam Winstanley [Irlande (pays)] ; Yu Zhou [Irlande (pays)] ; Su Yang [République populaire de Chine] ; Luo Yan [République populaire de Chine, Hong Kong] ; Liu Wenyin [République populaire de Chine, Hong Kong] ; Dave Elliman [Royaume-Uni] ; Mathieu Delalandre [France] ; Eric Trupin [France] ; Sébastien Adam [France] ; Jean-Marc Ogier [France]Source :
- International Journal of Document Analysis and Recognition (IJDAR) [ 1433-2833 ] ; 2007-03-01.
English descriptors
Abstract
Abstract: Performance evaluation is receiving increasing interest in graphics recognition. In this paper, we discuss some questions regarding the definition of a general framework for evaluation of symbol recognition methods. The discussion is centered on three key elements in performance evaluation: test data, evaluation metrics and protocols of evaluation. As a result of this discussion we state some general principles to be taken into account for the definition of such a framework. Finally, we describe the application of this framework to the organization of the first contest on symbol recognition in GREC’03, along with the results obtained by the participants.
Url:
DOI: 10.1007/s10032-006-0033-x
Affiliations:
- Espagne, France, Hong Kong, Irlande (pays), Royaume-Uni, République populaire de Chine
- Angleterre, Catalogne, Grand Est, Haute-Normandie, Lorraine (région), Nottinghamshire, Région Normandie
- Barcelone, Nottingham, Rouen, Villers-lès-Nancy
- Université de Nottingham
Links toward previous steps (curation, corpus...)
- to stream Istex, to step Corpus: 000B88
- to stream Istex, to step Curation: 000B81
- to stream Istex, to step Checkpoint: 001396
- to stream Hal, to step Corpus: 000689
- to stream Hal, to step Curation: 000689
- to stream Hal, to step Checkpoint: 003E02
- to stream Main, to step Merge: 005689
- to stream Main, to step Curation: 005543
Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title xml:lang="en">A general framework for the evaluation of symbol recognition methods</title><author><name sortKey="Valveny, E" sort="Valveny, E" uniqKey="Valveny E" first="E." last="Valveny">E. Valveny</name></author><author><name sortKey="Dosch, P" sort="Dosch, P" uniqKey="Dosch P" first="P." last="Dosch">P. Dosch</name></author><author><name sortKey="Winstanley, Adam" sort="Winstanley, Adam" uniqKey="Winstanley A" first="Adam" last="Winstanley">Adam Winstanley</name></author><author><name sortKey="Zhou, Yu" sort="Zhou, Yu" uniqKey="Zhou Y" first="Yu" last="Zhou">Yu Zhou</name></author><author><name sortKey="Yang, Su" sort="Yang, Su" uniqKey="Yang S" first="Su" last="Yang">Su Yang</name></author><author><name sortKey="Yan, Luo" sort="Yan, Luo" uniqKey="Yan L" first="Luo" last="Yan">Luo Yan</name></author><author><name sortKey="Wenyin, Liu" sort="Wenyin, Liu" uniqKey="Wenyin L" first="Liu" last="Wenyin">Liu Wenyin</name></author><author><name sortKey="Elliman, Dave" sort="Elliman, Dave" uniqKey="Elliman D" first="Dave" last="Elliman">Dave Elliman</name></author><author><name sortKey="Delalandre, Mathieu" sort="Delalandre, Mathieu" uniqKey="Delalandre M" first="Mathieu" last="Delalandre">Mathieu Delalandre</name></author><author><name sortKey="Trupin, Eric" sort="Trupin, Eric" uniqKey="Trupin E" first="Eric" last="Trupin">Eric Trupin</name></author><author><name sortKey="Adam, Sebastien" sort="Adam, Sebastien" uniqKey="Adam S" first="Sébastien" last="Adam">Sébastien Adam</name></author><author><name sortKey="Ogier, Jean Marc" sort="Ogier, Jean Marc" uniqKey="Ogier J" first="Jean-Marc" last="Ogier">Jean-Marc Ogier</name></author></titleStmt><publicationStmt><idno type="wicri:source">ISTEX</idno><idno type="RBID">ISTEX:32991B41E0527476CB97DB1BCBDB86BE899B70B1</idno><date when="2006" year="2006">2006</date><idno type="doi">10.1007/s10032-006-0033-x</idno><idno type="url">https://api.istex.fr/ark:/67375/VQC-BR7HLGG1-3/fulltext.pdf</idno><idno type="wicri:Area/Istex/Corpus">000B88</idno><idno type="wicri:explorRef" wicri:stream="Istex" wicri:step="Corpus" wicri:corpus="ISTEX">000B88</idno><idno type="wicri:Area/Istex/Curation">000B81</idno><idno type="wicri:Area/Istex/Checkpoint">001396</idno><idno type="wicri:explorRef" wicri:stream="Istex" wicri:step="Checkpoint">001396</idno><idno type="wicri:doubleKey">1433-2833:2006:Valveny E:a:general:framework</idno><idno type="wicri:source">HAL</idno><idno type="RBID">Hal:inria-00176899</idno><idno type="url">https://hal.inria.fr/inria-00176899</idno><idno type="wicri:Area/Hal/Corpus">000689</idno><idno type="wicri:Area/Hal/Curation">000689</idno><idno type="wicri:Area/Hal/Checkpoint">003E02</idno><idno type="wicri:explorRef" wicri:stream="Hal" wicri:step="Checkpoint">003E02</idno><idno type="wicri:doubleKey">1433-2833:2007:Valveny E:a:general:framework</idno><idno type="wicri:Area/Main/Merge">005689</idno><idno type="wicri:Area/Main/Curation">005543</idno><idno type="wicri:Area/Main/Exploration">005543</idno></publicationStmt><sourceDesc><biblStruct><analytic><title level="a" type="main" xml:lang="en">A general framework for the evaluation of symbol recognition methods</title><author><name sortKey="Valveny, E" sort="Valveny, E" uniqKey="Valveny E" first="E." last="Valveny">E. Valveny</name><affiliation wicri:level="3"><country xml:lang="fr">Espagne</country><wicri:regionArea>Centre de Visió per Computador, Edifici O, Campus UAB, Bellaterra (Cerdanyola), 08193, Barcelona</wicri:regionArea><placeName><settlement type="city">Barcelone</settlement><region nuts="2" type="region">Catalogne</region></placeName></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Espagne</country></affiliation></author><author><name sortKey="Dosch, P" sort="Dosch, P" uniqKey="Dosch P" first="P." last="Dosch">P. Dosch</name><affiliation wicri:level="3"><country xml:lang="fr">France</country><wicri:regionArea>LORIA, 615, rue du jardin botanique, B.P. 101, 54602, Villers-lès-Nancy Cedex</wicri:regionArea><placeName><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region><settlement type="city">Villers-lès-Nancy</settlement></placeName></affiliation><affiliation wicri:level="1"><country wicri:rule="url">France</country></affiliation></author><author><name sortKey="Winstanley, Adam" sort="Winstanley, Adam" uniqKey="Winstanley A" first="Adam" last="Winstanley">Adam Winstanley</name><affiliation wicri:level="1"><country xml:lang="fr">Irlande (pays)</country><wicri:regionArea>National University of Ireland, Maynooth, County Kildare</wicri:regionArea><wicri:noRegion>County Kildare</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Irlande (pays)</country></affiliation></author><author><name sortKey="Zhou, Yu" sort="Zhou, Yu" uniqKey="Zhou Y" first="Yu" last="Zhou">Yu Zhou</name><affiliation wicri:level="1"><country xml:lang="fr">Irlande (pays)</country><wicri:regionArea>National University of Ireland, Maynooth, County Kildare</wicri:regionArea><wicri:noRegion>County Kildare</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Irlande (pays)</country></affiliation></author><author><name sortKey="Yang, Su" sort="Yang, Su" uniqKey="Yang S" first="Su" last="Yang">Su Yang</name><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Department of Computer Science and Engineering, Fudan University, 200433, Shanghai</wicri:regionArea><wicri:noRegion>Shanghai</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">République populaire de Chine</country></affiliation></author><author><name sortKey="Yan, Luo" sort="Yan, Luo" uniqKey="Yan L" first="Luo" last="Yan">Luo Yan</name><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Department of Computer Science, City University of Hong Kong, Honk Kong</wicri:regionArea><wicri:noRegion>Honk Kong</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Hong Kong</country></affiliation></author><author><name sortKey="Wenyin, Liu" sort="Wenyin, Liu" uniqKey="Wenyin L" first="Liu" last="Wenyin">Liu Wenyin</name><affiliation wicri:level="1"><country xml:lang="fr">République populaire de Chine</country><wicri:regionArea>Department of Computer Science, City University of Hong Kong, Honk Kong</wicri:regionArea><wicri:noRegion>Honk Kong</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Hong Kong</country></affiliation></author><author><name sortKey="Elliman, Dave" sort="Elliman, Dave" uniqKey="Elliman D" first="Dave" last="Elliman">Dave Elliman</name><affiliation wicri:level="4"><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea>University of Nottingham, Nottingham</wicri:regionArea><orgName type="university">Université de Nottingham</orgName><placeName><settlement type="city">Nottingham</settlement><region type="nation">Angleterre</region><region type="région" nuts="1">Nottinghamshire</region></placeName></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Royaume-Uni</country></affiliation></author><author><name sortKey="Delalandre, Mathieu" sort="Delalandre, Mathieu" uniqKey="Delalandre M" first="Mathieu" last="Delalandre">Mathieu Delalandre</name><affiliation wicri:level="1"><country xml:lang="fr">France</country><wicri:regionArea>L3i Laboratory, La Rochelle University, Rochelle</wicri:regionArea><wicri:noRegion>Rochelle</wicri:noRegion><wicri:noRegion>Rochelle</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">France</country></affiliation></author><author><name sortKey="Trupin, Eric" sort="Trupin, Eric" uniqKey="Trupin E" first="Eric" last="Trupin">Eric Trupin</name><affiliation wicri:level="3"><country xml:lang="fr">France</country><wicri:regionArea>LITIS Laboratory, Rouen University, Rouen</wicri:regionArea><placeName><region type="region">Région Normandie</region><region type="old region">Haute-Normandie</region><settlement type="city">Rouen</settlement></placeName></affiliation></author><author><name sortKey="Adam, Sebastien" sort="Adam, Sebastien" uniqKey="Adam S" first="Sébastien" last="Adam">Sébastien Adam</name><affiliation wicri:level="3"><country xml:lang="fr">France</country><wicri:regionArea>LITIS Laboratory, Rouen University, Rouen</wicri:regionArea><placeName><region type="region">Région Normandie</region><region type="old region">Haute-Normandie</region><settlement type="city">Rouen</settlement></placeName></affiliation><affiliation wicri:level="1"><country wicri:rule="url">France</country></affiliation></author><author><name sortKey="Ogier, Jean Marc" sort="Ogier, Jean Marc" uniqKey="Ogier J" first="Jean-Marc" last="Ogier">Jean-Marc Ogier</name><affiliation wicri:level="1"><country xml:lang="fr">France</country><wicri:regionArea>L3i Laboratory, La Rochelle University, Rochelle</wicri:regionArea><wicri:noRegion>Rochelle</wicri:noRegion><wicri:noRegion>Rochelle</wicri:noRegion></affiliation><affiliation wicri:level="1"><country wicri:rule="url">France</country></affiliation></author></analytic><monogr></monogr><series><title level="j">International Journal of Document Analysis and Recognition (IJDAR)</title><title level="j" type="abbrev">IJDAR</title><idno type="ISSN">1433-2833</idno><idno type="eISSN">1433-2825</idno><imprint><publisher>Springer-Verlag</publisher><pubPlace>Berlin/Heidelberg</pubPlace><date type="published" when="2007-03-01">2007-03-01</date><biblScope unit="volume">9</biblScope><biblScope unit="issue">1</biblScope><biblScope unit="page" from="59">59</biblScope><biblScope unit="page" to="74">74</biblScope></imprint><idno type="ISSN">1433-2833</idno></series></biblStruct></sourceDesc><seriesStmt><idno type="ISSN">1433-2833</idno></seriesStmt></fileDesc><profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>Performance evaluation</term><term>Symbol recognition</term></keywords><keywords scheme="mix" xml:lang="en"><term>Performance evaluation</term><term>Symbol recognition</term></keywords></textClass><langUsage><language ident="en">en</language></langUsage></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Abstract: Performance evaluation is receiving increasing interest in graphics recognition. In this paper, we discuss some questions regarding the definition of a general framework for evaluation of symbol recognition methods. The discussion is centered on three key elements in performance evaluation: test data, evaluation metrics and protocols of evaluation. As a result of this discussion we state some general principles to be taken into account for the definition of such a framework. Finally, we describe the application of this framework to the organization of the first contest on symbol recognition in GREC’03, along with the results obtained by the participants.</div></front></TEI><affiliations><list><country><li>Espagne</li><li>France</li><li>Hong Kong</li><li>Irlande (pays)</li><li>Royaume-Uni</li><li>République populaire de Chine</li></country><region><li>Angleterre</li><li>Catalogne</li><li>Grand Est</li><li>Haute-Normandie</li><li>Lorraine (région)</li><li>Nottinghamshire</li><li>Région Normandie</li></region><settlement><li>Barcelone</li><li>Nottingham</li><li>Rouen</li><li>Villers-lès-Nancy</li></settlement><orgName><li>Université de Nottingham</li></orgName></list><tree><country name="Espagne"><region name="Catalogne"><name sortKey="Valveny, E" sort="Valveny, E" uniqKey="Valveny E" first="E." last="Valveny">E. Valveny</name></region><name sortKey="Valveny, E" sort="Valveny, E" uniqKey="Valveny E" first="E." last="Valveny">E. Valveny</name></country><country name="France"><region name="Grand Est"><name sortKey="Dosch, P" sort="Dosch, P" uniqKey="Dosch P" first="P." last="Dosch">P. Dosch</name></region><name sortKey="Adam, Sebastien" sort="Adam, Sebastien" uniqKey="Adam S" first="Sébastien" last="Adam">Sébastien Adam</name><name sortKey="Adam, Sebastien" sort="Adam, Sebastien" uniqKey="Adam S" first="Sébastien" last="Adam">Sébastien Adam</name><name sortKey="Delalandre, Mathieu" sort="Delalandre, Mathieu" uniqKey="Delalandre M" first="Mathieu" last="Delalandre">Mathieu Delalandre</name><name sortKey="Delalandre, Mathieu" sort="Delalandre, Mathieu" uniqKey="Delalandre M" first="Mathieu" last="Delalandre">Mathieu Delalandre</name><name sortKey="Dosch, P" sort="Dosch, P" uniqKey="Dosch P" first="P." last="Dosch">P. Dosch</name><name sortKey="Ogier, Jean Marc" sort="Ogier, Jean Marc" uniqKey="Ogier J" first="Jean-Marc" last="Ogier">Jean-Marc Ogier</name><name sortKey="Ogier, Jean Marc" sort="Ogier, Jean Marc" uniqKey="Ogier J" first="Jean-Marc" last="Ogier">Jean-Marc Ogier</name><name sortKey="Trupin, Eric" sort="Trupin, Eric" uniqKey="Trupin E" first="Eric" last="Trupin">Eric Trupin</name></country><country name="Irlande (pays)"><noRegion><name sortKey="Winstanley, Adam" sort="Winstanley, Adam" uniqKey="Winstanley A" first="Adam" last="Winstanley">Adam Winstanley</name></noRegion><name sortKey="Winstanley, Adam" sort="Winstanley, Adam" uniqKey="Winstanley A" first="Adam" last="Winstanley">Adam Winstanley</name><name sortKey="Zhou, Yu" sort="Zhou, Yu" uniqKey="Zhou Y" first="Yu" last="Zhou">Yu Zhou</name><name sortKey="Zhou, Yu" sort="Zhou, Yu" uniqKey="Zhou Y" first="Yu" last="Zhou">Yu Zhou</name></country><country name="République populaire de Chine"><noRegion><name sortKey="Yang, Su" sort="Yang, Su" uniqKey="Yang S" first="Su" last="Yang">Su Yang</name></noRegion><name sortKey="Wenyin, Liu" sort="Wenyin, Liu" uniqKey="Wenyin L" first="Liu" last="Wenyin">Liu Wenyin</name><name sortKey="Yan, Luo" sort="Yan, Luo" uniqKey="Yan L" first="Luo" last="Yan">Luo Yan</name><name sortKey="Yang, Su" sort="Yang, Su" uniqKey="Yang S" first="Su" last="Yang">Su Yang</name></country><country name="Hong Kong"><noRegion><name sortKey="Yan, Luo" sort="Yan, Luo" uniqKey="Yan L" first="Luo" last="Yan">Luo Yan</name></noRegion><name sortKey="Wenyin, Liu" sort="Wenyin, Liu" uniqKey="Wenyin L" first="Liu" last="Wenyin">Liu Wenyin</name></country><country name="Royaume-Uni"><region name="Angleterre"><name sortKey="Elliman, Dave" sort="Elliman, Dave" uniqKey="Elliman D" first="Dave" last="Elliman">Dave Elliman</name></region><name sortKey="Elliman, Dave" sort="Elliman, Dave" uniqKey="Elliman D" first="Dave" last="Elliman">Dave Elliman</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Lorraine/explor/InforLorV4/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 005543 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 005543 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Lorraine
|area= InforLorV4
|flux= Main
|étape= Exploration
|type= RBID
|clé= ISTEX:32991B41E0527476CB97DB1BCBDB86BE899B70B1
|texte= A general framework for the evaluation of symbol recognition methods
}}
| This area was generated with Dilib version V0.6.33. |  |